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Making Decisions About the Inclusion 
of Students With Disabilities in Large-Scale Assessments 



Overview 

This report is a summary of a meeting held in Washington, D.C. on March 9-10, 1994. 
The purpose of the meeting was to: 

• Discuss inclusion of students with disabilities in national and state 
large-scale assessments 

• Discuss adaptations in assessments or assessment procedures to 
accommodate students with disabilities 

• Attempt to arrive at agreement on a reasonable set of guidelines for 
making inclusion and accommodation decisions 

• Identify major technical and implementation issues that might be 
part of a federal research agenda on inclusion and accommodations 
in assessments 

In this report we provide background for the meeting, state the issues addressed, and 
summarize the major points of agreement reached. We propose a set of guidelines for 
making inclusion and accommodation decisions, and indicate the kinds of research 
questions that will need to be addressed over the next two to five years, to move the 
nation and states forward in this important area. 

Much of the discussion at this meeting focused on the National Assessment of 
Educational Progress (NAEP). The challenges faced in making decisions about 
inclusion and accommodations in NAEP in many ways parallel those faced by states. 
Thus, the content of this report and the recommended guidelines also have applicability 
for state large-scale assessments. 



Background 

Students with disabilities have been excluded to an unreasonable extent from large scale 
assessment programs at national, state, and local levels. There are differential 
participation rates across states. For example, in the 1990 Tri al State NAEP, exclusion 
rates ranged from 33% to 87% of students with disabilities. 

States use a variety of guidelines in making decisions about who to include in their state 
assessments, and they allow differing kinds of accommodations in assessment. The 
guideline recommended by NCES for making decisions about who to include in NAEF 
has been adopted by many states for their own assessment programs. It reads: 
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Students on Individual Educational Plans (EPS) may be excluded if 
the student is mainstreamed less than 50 percent of the time in 
academic subjects and is judged to be incapable of taking part in the 
assessment or the IEP team has determined that the student is 
incapable of taking part meaningfully in the assessment. 

Many factors lead to the exclusion of students with disabilities from'large-scale 
assessments. These include: 

• Use of vague guidelines that allow local decisions to be made about 
the participation of students who are on DBPs 

o Differential and inconsistent implementation of guidelines 

• Variability in monitoring the extent to which the intent of the 
guideline is followed 

• Sampling plans that systematically exclude students who are in 
separate schools and students who are not in graded programs 

• An unwillingness to make accommodations in assessment 
materials and procedures that would enable more students with 
disabilities to participate 

• Altruistic motivation to lessen emotional distress to students who 
are not expected to do well 

• Incentives created by the desire to have a school or state look good 
in comparison to others in the state or nation 

Importance 

Educational reform in the 1990s emphasizes assessment as a means to measure progress 
toward goals. Students who are left out of assessments tend not to be considered 
during reform efforts. And, estimates of performance for states on such assessments as 
NAEP are of questionable comparability because of differential participation rates. 
Policymakers have inaccurate or incomplete data for making educational policy 
decisions. And, educators, businesses, and others have poor information on how we as 
a nation or individual states are doing in educating students with disabilities. 



Synthesis Report 13 



The March Meeting 

A meeting was held in Washington, DC on March 9-10, 1994 and was attended by those 
individuals listed at the end of this report (see Appendix A). Prior to the meeting, 
participants were provided with copies of the following materials: 

Guidelines for Inclusion of Students wi3i Disabilities in Large Scale 
Assessments, prepared by the National Center on Educational 
Outcomes (NCEO) 

Assessing Students with Disabilities and Limited English Proficiency, 
prepared by the National Center for Education Statistics (NCES) 

Also available to meeting participants were the following NCEO reports: 

Inclusion of Students with Disabilities in National and State Data 
Collection Programs (Technical Report 2) 

Testing Accommodations for Students with Disabilities: A Review of 
the Literature (Synthesis Report 4) 

Views on Inclusion and Testing Accommodations for Students with 
Disabilities (Synthesis Report 7) 

In the NCEO Guidelines document, participants were provided with materials that 
included background information, a rationale for developing guidelines, alternative 
approaches, and a set of recommendations for practice (see Appendix B). The 
recommendations were developed by staff at NCEO based on written input from 
experts (see NCEO Synthesis Report 7) and by others who reacted to our initial drafts of 
recommendations. Highlights of the pre-meeting recommendations are provided here. 




Inclusion: Including students with disabilities in large scale assessments needs to occur at 
three points: (1) instrument development (2) instrument administration, and (3) reporting 
of results* 



Accommodation* and Adaptations; Not all students with disabilities will need 
accommodations during assessments. But modifications in assessments should be used 
when needed to increase the number of students with disabilities who can take tests. 
Accommodations and adaptations that teachers currently use with students during 
instruction and that are permitted by society should be used during assessments. 
Modifications that may raise questions about the technical characteristics of measures should 
be studied. As new technologies and procedures for accommodations and adaptations are 
developed, they should be included in the array of possible accommodations and adaptations 
for instruction and testing. 



ERLC 



3 



Synthesis Report 13 



Monitoring: Monitor adherence to the intent of the guidelines by making sure that no 
student is excluded who could participate if accommodations and adaptations were used. 
Do this by requiring a specific person in the district to sign off for each student who does not 
participate in the regular assessment and by having the student complete an alternative 
assessment or having someone provide information about the student 

The meeting agenda also is appended at the end of this report (see Appendix C). We 
spent the first part of the meeting on guidelines for inclusion, the second part on 
guidelines for accommodation/ test adaptation and monitoring, and the third part on 
research needs related to these topics. There was general agreement that the topics of 
inclusion and accommodations represent a challenge for those engaged in title 
construction, administration, and interpretation of large-scale assessment programs* 
And, it is not crystal dear how to proceed in addressing these important challenges. 
Yet, there was a strong commitment to do so among the leaders present, and some very 
good ideas about how to proceed were discussed. The following is a summary of the 
major points made during the March 9-10 discussions. 

1. It is important to state explicitly the assumptions that underlie our guidelines for 
making inclusion and accommodation decisions. The assumptions include: 

• Accuracy and fairness should characterize the assessment 

• Assessment should provide information on students with disabilities 

• Assessment procedures should be sensitive to the needs of students with 
disabilities 

• Assessment should make clear that the same high standards are expected of all 
students 

• Assessment should be characterized by practicality and cost effectiveness 

• Assessment should be consistent with students' instructional programs and 
accommodations. 

2. We have a good understanding of the magnitude of exclusion and the variability 
in both inclusion and accommodation practices. The American Institutes for 
Research is now conducting a follow-up study of students excluded from the 1994 
NAEP Trial State Assessment in Reading. This study should provide considerable 
information on the characteristics of excluded students, the reasons they are being 
excluded, and the extent to which they can be assessed. At the conclusion of this 
study, we should know more about the kinds of accommodations that would be 
necessary for excluded students to participate in large-scale assessments. 
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3. The challenges that are faced by NAEP personnel in making inclusion and 
accommodation decisions are very similar to the challenges being faced at the state 
level Thus, for the most part, the issues for NAEP and other large-scale state 
assessments are similar. 

4. There is a need to be very clear about the characteristics of students we are talking 
about when we t alk about students with disabilities. There are 13 federal 
categories of students with disabilities, and within each category students 
demonstrate a wide range of skills and abilities. It is estimated that as many as 
85% of the nearly 5 million students who are now considered eligible for special 
education services (i.e., they are on DSPs) could take large-scale tests without 
adaptations or accommodations. These students include many of the students 
with learning disabilities, emotional or behavioral disabilities and some with 
mental retardation. One factor that limits the participation of some of these 
students in NAEP is the lack of items appropriate for low-functioning students. Of 
course, not all students with disabilities are low functioning. 

5. We need to think carefully about how the scores on students with disabilities 
would be used. In our discussions about inclusion and accommodation in testing 
we need to differentiate the purpose of the assessment. Data obtained from large- 
scale assessments are used to make many different kinds of decisions. Most of 
these are not high-stakes decisions for individual students. Most are used for 
descriptive purposes. There is a need to know how students with disabilities are 
doing in the nation's schools. And, if NAEP is the nation's "report card," then 
students with disabilities need to be included and information on their 
performance needs to be reported. 

6. The current N/EP guideline for making inclusion/exclusion decisions is 
problematic in at least two ways. One way is its use of a percentage of Mme in the 
mainstream setting. The second way is in its reliance on the "IEP team" (or some 
designated person) to make decisions about "meaningful participation" in 
assessments. 

(a) Percentage of time in the mainstream is not a good indicator of a 
student's instructional program, level of skill development, or 
ability. There are too many other factors that enter into 

mains treaming decisions. 

(b) The "IEP team" allows too much slippage in the team decision- 
making process. Frequently, the IEP terminology is interpreted to 
mean that any student on an IEP should be excluded from testing. 
Sometimes decisions are made solely on the basis of the student's 
category of disability. It is not a good idea to encourage IEP teams 
to decide whether students should take tests. 
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7. Because NAEP is an assessment designed to describe the status of students in our 
nation, it is important to include as many students as possible. Recommendations 
for revising the terminology in the NAEP "exclusion 11 guidelines addressed the two 
problematic aspects of the guidelines. 

(a) Rattier than using a percentage of time measure, a better indicator 
would be correspondence between the content the tesc is intended 
to measure and the type of curriculum for the students. Students 
who are working toward outcomes other than those measured by 
the assessment (e.g., functional skills) should participate in an 
alternative form of assessment The type of curriculum rather than 
the setting should be the factor that determines the nature of 
assessment 

(b) Rather than referring to the IEP, it would be better to identify skills 
needed to take the assessment School building administrators 
could be provided with a checklist of factors to consider in making 
inclusion/exclusion decisions. 

8. Students with disabilities need to be included during the item development 
process for NAEP and state assessments. Their participation will provide test 
developers with information on ceiling and floor effects, and assist them in the 
identification of items that aie problematic without adaptation. 

9. The notion of assigning zero scores to students who are excluded from testing was 
not considered acceptable. The possible use of "imputation of scores" needs to be 
carefully researched before being implemented within large-scale assessments of 
students with disabilities. Care must be taken riot to assume that students with a 
particular disability label (e.g., learning disability - LD) are homogeneous. For 
such heterogeneous groups, use of traditional characteristics may be inappropriate. 
Further, to be consistent with the principle of inclusion, students with disabilities 
must be actually assessed. Therefore, imputation of scores and score distribution 
characteristics from nonassessment data should be discouraged. 

10. There are three types of students with disabilities with regard to assessment: those 
who can take large-scale assessments with no accommodations, those who can be 
included with adaptations/accommodations, and those who need alternative 
assessments. The first group can be included now, and some are included already. 
Many in the second group can be included with very minor adaptations (e„g., 
testing in separate setting) that should not interfere with test validity. However, 
significant work needs to be devoted to developing accommodations for those in 
the second group who cannot now be included and to developing alternative 
methods for gathering data on the performance and achievement of stud/ants who 
are unable to participate in regular assessments. 
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11. Students with disabilities must be included in all reporting of results- Scores could 
be reported for subgroups of students: students without disabilities, students with 
disabilities who took the test with no accommodations, students with disabilities 
who took the test with accommodations, and students with disabilities who took 
an alternative measure (assuming one were developed)- Data could then be 
aggregated and reported separately by subgroup or for combinations of 
subgroups. This was viewed as a temporary solution to the inclusion and 
adaptation challenges. It was hoped we can move quickly to the day when 
accommodations become invisible means to enable students to participate in 
assessments, and data could be reported for all students in aggregate, 

12. There ought to be an immediate move to (a) encourage school districts to include 
in NAEP more students with disabilities who do not need accommodations, and 
(b) permit modifications that should not affect the validity of the test (e.g., testing 
in a separate room, use of magnifying glasses, etc.). It was recognized that testing 
in varied settings would have cost implications. It was suggested that funds be 
made available or reallocated from funds now directed toward general NAEP 
testing. ETS currently estimates the cost of such modifications on administration 
of the GRE, SAT, and other assessments. Estimates of the costs of test 
modifications at ETS are based on past experience in providing them over many 
years. 

13. There is a need to monitor exclusion of students with disabilities. Large-scale 
assessments employ monitors to ensure that standardized procedures are 
followed. These monitors couJd also focus on sampling and exclusion decisions. 
The monitors should be tough and ask lots of questions about excluded students. 
It was also suggested that sanctions be applied to schools or states with large 
exclusion rates. As is done with states that do not have high encugh school 
participation rates in NAEP, those with exclusion rates that are too high would not 
be included in reports of results. 

14. There is an immediate need for follow-up studies of students excluded from 
participation in NAEP and other forms of large-scale assessment. 

15. Some attention should be given to identification and delineation of factors that 
serve as incentives to excluding students with disabilities from testing. Then, 
efforts should be made to remove the incentives. 

Recommended Guidelines for Making Inclusion Decisions 

1. Include students with disabilities when trying out items in order to 
identify problematic item formats and the need for more items at the lower 
end, for example. In this way, instruments can be modified during the 
development phase (e.g., items dropped, modified, or added) to allow 
greater numbers of students with disabilities to participate meaningfully. 
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2. Include all students with disabilities in taking some form of the 
assessment When a sampling procedure is used for an assessment, the 
sample must be representative of all students* 

• Allow partial participation in an assessment* Some assessments have 
components that could be completed by an informed respondent 
Include students with disabilities in these components), even if they 
cannot personally complete a test 

• Use an alternative assessment for some students* Those students 
whose curriculum differs significantly from the content of the 
assessment should be assessed on an alternative assessment This 
decision about whether a student can participate in the regular 
assessment is to be made by responding to the questions on a checklist 
A possible checklist is included in Appendix D. 

3. Include students with disabilities in the reporting of results* Data on 
the performance of all students are needed* Therefore, scores must be 
reported for all students. Reports of results from students taking 
alternative assessments and from information provided by informed 
respondents should be included in these reports* 

Recommended Guidelines for Making Accommodation Decisions 

Not all students with disabilities will need accommodations during assessments. But 
modifications in assessments should be used when needed to increase the number of 
students with disabilities who can take tests* Accommodations and adaptations that 
teachers currently use with students during instruction and that are permitted by 
society should be used during assessments* Initially, it is possible to use modifications 
that: 

• Make a student more comfortable and secure in the test setting (for 
example, using carrels, separate room administrations, etc.) 

• Do not destroy the validity of measures (for example, amplification, 
magnification, large print version, Braille version, augmentative 
communication, sign language, word processor, etc.) 

Other modifications that may raise questions about the technical characteristics of 
measures should be studied. These other types of accommodations and adaptations 
include: 

• Presentation alterratives - audiocassette, oral administration 

• Response alternatives — dictate to scribe, Braille writer 
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• Setting alternatives — individual administration, hospital 
administration 

• Timing/ scheduling alternatives — extended time, multiple test 
sessions 

As new technologies and procedures for accommodations and adaptations are 
developed, they should be subjected to validation research with an eye for inclusion in 
the array of possible accommodations and adaptations for instruction and testing. 

Recommendations for Monitoring 

Monitor adherence to the intent of the guidelines, giving strong incentives so that no 
student is excluded who could participate if accommodations and adaptations were 
used. Do this by requiring a specific person in the district to sign off for each student 
who does not participate in the regular assessment and by requiring that the student 
complete an alternative assessment or by requiring that someone provide information 
about the student's achievement. In addition: 

• Conduct and report results of follow-up studies of currently 
excluded students to verify that these students could not 
participate in the assessment with reasonable modifications. 

• Remove incentives for exclusion by not reporting the results for 
states that exclude a defined percentage students with disabilities. 

• Set up a panel to review requests for new forms of testing 
modifications so that decisions can be made about the 
reasonableness of the requested modifications, or a decision made 
about the need for research. 



Where Do We Go from Here? 

The following actions were suggested: 

1. Those in attendance at the meeting agreed to respond within one week to the 
content of the NCES discussion draft on assessing students with disabilities and 
limited English proficiency. 

2. We need to keep working toward inclusion of students with disabilities in the 
nation's report card. NCES personnel should strongly encourage states to include 
students with disabilities in NAEP. They should be serious about the "when in 
doubt, include" notion. OSERS might prepare a letter that would be included with 
the NAEP materials indicating the importance of including students with 
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disabilities. NCES will monitor the extent to which students with disabilities are 
included. 



3. NCEO should make available to the National Assessment Governing Board 
(NAGB), the Education Information Advisory Committee (EIAC), and the 
Association of State Assessment Personnel (ASAP) a copy of this summary of the 
March 9-10 meeting. NCEO personnel should strive to meet with representatives 
of these groups to present a summary of the recommendations from this meeting. 

4. NCEO should communicate the results of this meeting to the American 
Psychological Association Committee on Psychological Tests and Assessments 
(COPTA) and solicit their reaction. 

5. OERI and OSEP should include in their research priorities a set of research 
questions like those generated at this meeting. These questions focused on 
demographics,, technical issues, implementation concerns, outcomes/consequences 
of assessment, and policy issuer The specific questions, in their modified form 
(based on meeting discussions), are presented in Appendix E. 
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APPENDIX B 

NCEO Document "Guidelines for 
Inclusion of Students with Disabilities in 
Large Scale Assessments" 

[This document was provided to participants prior to the 
meeting as a stimulus for discussion] 
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Guidelines for Inclusion of Students with Disabilities in 
Large Scale Assessments 



Background 

Students with disabilities have been excluded to an unreasonable extent from large scale assessment 
programs at national, state, and local levels. Large scale assessment programs of note include the 
National Assessment of Educational Progress (NAEP), our nation's "report card," state assessment 
programs, and school district assessments that are used to describe the performance of all students in a 
given location (the nation, a state, or a school district). 

One of the implications of this exclusion practice is that students who are left out of assessments tend 
not to be considered during reform efforts. Another is that estimates of performance for states on such 
assessments as NAEP are not comparable because of differential participation rates (e.g., 1990 Trial 
State NAEP exclusion rates from 33% to 87% of students with disabilities % 

A common type of guideline is the one used by NAEP. This guideline has been adopted by many states 
for their own assessment programs. It reads: 

Students on Individual Educational Plans (EEPs) may be excluded if "The student is mainstteamed 
less than 50 percent of the tune in academic subjects and is judged to be incapable of taking part in 
the assessment or the EEP team has determined that the student is incapable of taking part 
meaningfully in the assessment" 

This guideline now has been the target of much criticism. But, of course, the guideline is not the sole 
source of exclusion. There are actually many factors that underlie the exclusion of students with 
disabilities from large scale assessments. They include: 

• The use of vague guidelines that allow local decisions to be made about the 
participation of students who are on Individualized Educational Plans (IEPs) 

• The differential implementation of guidelines 

• The failure to monitor the extent to which the intent of the guidelines is followed 

• Sampling plans that systematically exclude students who are in separate schools and 
students who are not in graded programs 

• An unwillingness to make accommodations in assessment materials and procedures 
that will enable some individuals to participate 

• An altruistic motivation to lessen the emotional distress to the student who is not 
expected to perform well 



Rationale for Developing Guidelines 

An underlying premise is that large scale assessment programs should include as many students with 
disabilities as possible without destroying desired technical characteristics, given the purpose of the 
assessment A consistent set of guidelines is needed It should have three components: 

• Guidelines for inclusion 

• Guidelines for accommodations and adaptations 

• Monitoring system to ensure adherence to guidelines 
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Alternative Approaches to Including Students with Disabilities 

Not all large scale assessment programs use the same approach to the inclusion of students with 
disabilities. States, in particular, vary considerably in both the guidelines they have for making 
decisions about the participation of students with disabilities in assessments and for determining what 
accommodations and adaptations are used during assessments. Furthermore, several individuals have 
considered the issues and made recommendations about best practice. The following array of 
approaches, which is neither an all-inclusive list nor a list of mutually exclusive alternatives, provides a 
basis for constructing an approach that might be used for all large scale assessments* 

Alternative 1 : Exclude any student who is on an IEP, using the documentation of success on 
the IEP as a substitute for performance on die assessment 

Alternative 2 : Include students who are in X mainstream classrooms for Y ~ 

2g: Define the X as a type of mainstream class (usually, academic) 
2fr Define the Y as a percentage of school time (usually, 50%) 
2g: Define the Y as a number of classes 
2& Define the Y as a number of credits 

Alternative 3 : Include an individual student based on the opinion of 

3& IEP team (with or without requirement that it be written on the IEP) 
2b: T> acher (special education or other teacher) 
2e: School administrator 
2d: Parent 
2& Student 

Alternative 4 : Make decisions about participation based on the category of disability 

(e.g,, students with learning disabilities participate, students with significant 
visual impairments do not participate, etc*) 

Alternative 5 : Make decisions about participation based on exposure to the content of the 
curriculum assessed 

Alternative 6 : Make decisions about participation based on the extent to which the test will 
yield a valid and reliable measure of the student's performance 

Alternative? : Make decisions about participation based on the extent to which 
accommodations can be made without undue cost 

Alternative8 : Include all students in the accountability system, with most in the regular 
large scale assessment and the remaining X% percentage of students with 
disabilities in an alternative accountability system 

Sa: Use the 2% figure for X%, also designating that this should include only 
those students with the more severe cognitive disabilities (2% based on 
KY guidelines) 

Sfc: Develop an alternative accountability system based on IEPs, portfolios, or 
other strategies for the 2% 

Alternative 9: Give all excluded students a "0" on the assessment and include them within 
the denominator when calculating averages, etc* 

Each of these alternatives has advantages and disadvantages associated with it Some rely too strongly 
on the opinion of one or more individuals whose opinions may be biased by unrelated issues* Some are 
too arbitrary, and not linked to the way students are currently served in schools* It is possible that legal 
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issues could make some options problematic. For example, if a large scale assessment has consequences 
for die student, issues of access and accommodations will have to be addressed Other considerations 
impinge on these alternatives as well. For example, many large scale assessment programs do not now 
make adequate differentiations among student performance at the lower end of performance. With the 
inclusion of more students who typically have performed at the lower end of the scale on large-scale 
assessments, it is likely that there will be a need for greater differentiation at this lower end. It is 
important to consider all of these factors in interaction when constructing the best approach. 



Recommended Practice 

Based on interactions with numerous policymakers, assessment personnel, disability advocates, and 
others, the following is recommended as a set of guidelines for large scale assessment programs to use to 
include students with disabilities in their assessments. 

Inclusion: Including students with disabilities in large scale assessments needs to occur at 
three points: (1) instrument development, (2) instrument administration, and (3) 
reporting of results. 

1 . Include students with disabilities when trying out items in order to identify 
problematic item formats and the need for more items at the lower end, for example. 
In this way, instruments can be modified during the development phase (e.g., items 
dropped modified, or added) to allow greater numbers of students with disabilities to 
participate meaningfully. 

2. Include all students with disabilities in "taking" some form of the assessment. 
When a sampling procedure is used for an assessment, the sample must be 
representative of all students. 

• Allow partial participation in an assessment. Some assessments have 
components that could be completed by an informed respondent Include students 
with disabilities in these components), even if they cannot take a test 

• Use an alternative assessment for some students. Allow up to 2% of the 
student population or the population sample to participate in an assessment that is 
developed as an alternative to the regular assessment These students should be 
those with the most severe cognitive disabilities. As a first step in setting up an 
alternative accountability system for these students, require the completion of a 
form containing functioning level information beyond that typically required on 
"excluded student" forms in large scale assessment 

3. Include students with disabilities in reporting of results. Data on the performance 
of all students are needed. Therefore, scores must be reported for all students. 
Reports of results from students taking alternative assessments and from information 
provided by informed respondents should be included in these reports. If a student is 
excluded from testing for any reason, that student should be given a score of zero. 

Accommodations and Adaptations : Not all students with disabilities will need 
accommodations during assessments. But modifications in assessments should be used when 
needed to increase the number of students with disabilities who can take tests. 
Accommodations and adaptations that teachers currently use with students during instruction 
and that are permitted by society should be used during assessments. Initially, it is possible 
to use modifications that: 

• Make a student more comfortable and secure in the test setting (for example, 
using carrels, separate room administrations, etc.) 
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• Do not destroy the validity of measures (for example, amplification, magnification, 
large print version, Braille version, augmentative communication, sign language, 
word processor, etc.) 

Other modifications that may raise questions about the technical characteristics of measures 
should be studied For example, other types of accommodations and adaptations include: 

• Presentation alternatives - audiocassette, oral administration 

• Response alternatives — dictate to scribe, Braille writer 

• Setting alternatives — individual administration, hospital administration 

• Timing/scheduling alternatives - extended time, multiple test sessions 

As new technologies and procedures for accommodations and adaptations are developed, 
they should be included in the array of possible accommodations and adaptations for 
instruction and testing. 

Monitoring: Monitor adherence to the intent of the guidelines by making sure that no 
student is excluded who could participate if accommodations and adaptations were used Do 
this by requiring a specific person in the district to sign off for each student who does not 
participate in the regular assessment and by having the student complete an alternative 
assessment or having someone provide information about the student In addition: 

• Conduct follow-up studies of excluded students to verify that these students could 
not participate in the assessment with reasonable modifications, and report the results 
of the follow-up studies. 

• Remove incentives for exclusion by assigning zero scores to all students who are 
excluded from assessments. 

• Set up a panel to review requests for new forms of testing modifications so that 
decisions can be made about the reasonableness of the requested modifications, or a 
decision made about the need for research. 
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March 9-10 Meeting Agenda 
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Agenda 



Working Meeting 
March 9-10, 1994 

Guidelines for Inclusion and Accommodations in 
Large Scale Assessment Programs 



March 9. 1994 
9:00-10:00 



10:00-10:15 

10:15-12:00 

12:00-1:00 

1:00-2:45 

2:45-3:00 

3:00-4:00 



Opening Comments 

Jim Ysseldyke, NCEO 
Emerson Elliott, NCES 
Tom Hehir, OSEP 

Introduction of Participants 
Break 

Discussion of Guidelines for Making Inclusion Decisions 
Lunch 

Discussion of Guidelines for Making Accommodation Decisions 
Break 

Brainstorm Ideas for Research on Technical Consequences of Test 
Modifications 



March 10. 1994 

9:00-10:00 Presentation of Framework Derived from Brainstormed Ideas 
10:00-12:00 Discussion of Research Framework 

12:00 - 1:00 Lunch 

(Summary of Guidelines for Inclusion and Accommodations, and 
Overview of Research Needs) 
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APPENDIX D 

Possible Checklist for 
Making Decisions on Inclusion of 
Individual Students with Disabilities 

[NAEP is used as an example in this checklist] 



Note: This checklist was developed as an example of 
what might be used. It was developed without first-hand 
knowledge of NAEP items. 

In addition, certain assumptions were made when 
this checklist was compiled. The primary assumption is 
that the initial pool for sampling would include all 
students, even those in separate rooms and separate 
buildings. The recommended steps for achieving this are 
as follows: 

1. Make a roster of all students in your school attendance 
area. This should include students who are attending 
separate schools (such as schools for students who are 
deaf, residences for students with psychiatric 
disorders, etc.) 

2. Do your selection of the NAEP sample using this 

roster. 

3. Make a list of students you believe should be excluded 
or for who accommodations should be made. 

4. Complete the attached forms for each student. 

5. Reach your decision and record it. 
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NAEP Inclusion Checklist 



Student Name 
School 



Directions: Answer the following questions for student identified above. Be sure to 
complete all sections of the worksheet 

Section I: Assessment Requirements 

1. Can the student work independently? [ ] Yes [ ] No 

2. Can the student work with 25 to 30 

other students in a quiet setting? [ ]Yes [ ]No 

3. Can the student work continuously 

for 20 to 30 minute periods? [ ]Yes [ ]No 

4. Can the student listen and follow 
oral directions given by an adult or an 

audiotape? [ ] Yes [ ]No 

5. Can the student use paper and pencil 
to write short-answer or paragraph length 

responses to open ended questions? [ ]Yes [ ]No 

6. Can the student understand and answer 

questions in a multiple choice format? [ ]Yes [ ]No 

7. Can the student read at a grade 

level without special arrangements? [ ]Yes [ ]No 



Note: Questions regarding any unique requirements of the particular content area, 
(i.e., reading, mathematics, science) would be included here as well 



Section II: Testing Accommodations and Adaptations 

Directions: If answering "No" to any question above, go to the attached Accommodations 
Checklist and follow its directions. If answering "Yes" to all questions above, proceed to 
Section III and select the first option. 



Section DDI: Decision Summary 

Directions: Based on your considerations, select one of the following recommendations. 
When in doubt, always choose in favor of inclusion. 

[ ] The student should be given the NAEP Assessment without any special 
accommodations. 

[ ] The student should be given the NAEP Assessment with the accommodations 
identified on die attached Accommodation Checklist 

[ ] The student should be given an alternative assessment 
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Adaptations and Accommodations Checklist 

Directions: To address those areas answered "No" in Section I, consider the following 
list of adaptations or accommodations, and do sms. of the following: 

1) Check those accommodations that would allow the student to successfully complete 
theasswsment (Remember to select only the minimal number of accommodations fliat 
would be necessary.) Then proceed to Section m and select the second option. 



■or 



SiS^tiOT 6 " NOne ° f Ae ab ° VeH Statement 711611 P***** to Scction m > select the 



Presentation Accommodations 

[ ] Braille edition of test 

[ ] Use of magnifying equipment 

[ ] Large-print edition of test 

[ ] Oral reading of directions 

[ ] Signing of directions 

[ ] Interpretation of directions 

[ ] Other 



Response Accommodations 

[ ] Maridng response in test booklet 

[ ] Using template for responding 

[ ] Pointing to respond 

[ ] Giving response orally 

[ ] Giving response in sign language 

[ ] Using typewriter for responding 

[ ] Using computer for responding 

[ ] Receiving assistance and interpretation with responses 

L J Other 



Setting Accommodations 

[ ] Testing alone in test carrel 

[ ] Testing with small group 

[ ] Testing at home 

[ ] Testing in special education/language classroom 

[ J ■ Other 



Timing Accommodations 

[ ] Extended time 
[ ] Shorter version of test 



[ ] More frequent breaks during testing 
[ ] Extended testing sessions over several davs 
[ ] Other J 



[ ] None of the above accommodations would assist the student in 
successfully completing the exam. 
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Possible Research Questions 
March 9-10, 1994 

(Letters in parentheses indicate the categories into which questions 
were organized during the meeting: C = Consequences, D = Descriptive, 
I = Implementation, P = Policy, T = Technical) 

Exclusion P nliey/Decisinn Making 

1 . What are the incentives that drive inclusion/exclusion of students with 

disabilities? (D) ^ xTABm 

2. What are the characteristics of students who are excluded from NAEP? \u) 

3. What does an assessment system that includes all kids look like? (T) 

4. What is the degree of assessibility of students who have IEPs and LEPs (or 
misclassified students)? (T) 

5. What is the operational and technical feasibility of including students from 
ungraded programs and special facilities, and youth out of school? (I) 

6. What are the students' perceptions of the testing experience? (C) 

7. What are characteristics of kids excluded? (P) 

8. What is an acceptable exclusion level? What are the merits and limitations of 
specific levels? (P) 

Adaptation/Accommo dations 

1. What types of accommodations are made in the work place (especially technology) 
and what are their effects on assessment? (D) 

2. What types of instructional accommodations are made for students with 
disabilities? (D) . 

3. What is the relationship between testing and instructional accommodations? (U) 

4. What are the effects of accommodations and adaptations on test validity for all 
students by content domain? (T) 

5. What are the effects of using trained versus non-trained staff and/or internal 
versus external staff (scribes, etc) on NAEP results? (T) 

6. Would using accommodations be incentives for schools to increase participation 
rates of students with disabilities? Or are some (e.g., longer time) disincentives? 

7. What are the costs of implementing accommodations, alternative assessments, and 
imputing data? (I) 

8. What are the characteristics of students who can take the test without 
accommodations? (C) 

Alternate Assessment 

1 . What are the effects of out of level testing? (T) 

2. Which types of alternative assessments make sense for students with disabilities, 
and which ones should be used in particular situations? (T) 

3. Are there ways to use portfolio assessments to make statements about what 
students know and can do? And how does this compare to NAEP?' (T) 
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4. To what extent is there a need for out of level testing for students with disabilities? 
Monitoring 

1. What are the monitoring strategies being used? (D 

2. What are the effects of different types of monitoring strategies? (I) 

3. What are the possible effects of negative strategies for monitoring NAEP? (C) 

Reporting/IJsy 

1. In what ways do differential patterns of inclusion/exclusion affect comparability 
of state NAEP, NAEP, and IAEP? (T) 

2. What are the assessment implications of Goals 2000, and what is their impact for 
students with disabilities? (I) v 

3. Are we getting a clearer picture about total population.- information that will 
help us make decisions that help kids? (C) 

4. Should performance standards be the same or different for students with 
disabilities? (P) 



Technical Asp ff( 



stribuHon P roperty 



2. 
3. 



What are some appropriate technical approaches for sparse matrices and 
probability sampling? (T) 

How can we get population estimates for groups of students? (T) 
Can we use more extensive assessments of special education populations to help 
make imputations for students with disabilities? (T) 
4. What is the impact of changing NAEP (through greater inclusion and/or use of 
accommodations) on trend analyses? (T) 

Validity of Interpigtatinns 

1. What are the effects of various accommodations (including time extensions) on 
score meaning? (include identifying the accommodations that have no effect) (T) 

2. What are the effects of standard NAEP tasks (performance or not) on score 
meaning for students with disabilities? To what extent does test bias occur when 
tests developed for general populations are used with special populations? (T) 
What is the auricular validity of NAEP for students with disabilities? (T) 



4. 
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